Skip to content

test: spike Vitest browser mode as WTR replacement#1299

Merged
iOvergaard merged 15 commits intomainfrom
v2/spike/65042-vitest
Feb 27, 2026
Merged

test: spike Vitest browser mode as WTR replacement#1299
iOvergaard merged 15 commits intomainfrom
v2/spike/65042-vitest

Conversation

@iOvergaard
Copy link
Copy Markdown
Contributor

@iOvergaard iOvergaard commented Feb 26, 2026 •
edited
Loading

Summary

Spike for AB#65042 — evaluate Vitest as a replacement for Web Test Runner.

Full migration of all 80 component test files from @open-wc/testing + WTR to vitest-browser-lit + Vitest 4 browser mode.

  • All 809 tests pass (3 skipped), across 80 test files
  • Removes all @open-wc/testing and WTR dependencies
  • CI runs all 3 browsers (Chromium + Firefox + WebKit), local dev runs Chromium only — same as WTR setup

Performance (local, Chromium only, same machine)

Runner Run 1 Run 2 Run 3 Tests
WTR 18s 21s 21.5s 807 passed
Vitest 4 8.5s 14.6s 9.7–10.8s 809 passed

Vitest is roughly ~2x faster on warm runs. The variance comes from Vite's dependency pre-bundling cache — cold starts are slower, cached runs are fast. Actual test execution is consistently ~5s in Vitest.

Key findings

WTR Vitest
Runner @web/test-runner + Mocha + Chai vitest + @vitest/browser
Rendering @open-wc/testing fixture() vitest-browser-lit render()
Assertions Chai + plugins Vitest (Jest-compatible)
a11y @open-wc/testing .accessible() axe-core direct + custom matcher
Mouse interaction @web/test-runner-commands sendMouse @vitest/browser/context userEvent.click()
Shadow DOM assertions @open-wc/testing .shadowDom.toBe() No equivalent — test behavior instead
Browsers (local) Playwright (Chromium) Playwright (Chromium)
Browsers (CI) Playwright (Chromium + Firefox + WebKit) Playwright (Chromium + Firefox + WebKit)

Migration patterns

  • render(html...).container.querySelector('tag')! replaces fixture(html...)
  • await element.updateComplete replaces elementUpdated(element)
  • vitest-axe does not work in browser mode — use axe-core directly with a custom matcher
  • Mocha done() callbacks → async/await with Promises
  • Chai .to.equal() → .toBe(), .to.have.property() → .toHaveProperty(), etc.
  • UUITestMouse now uses userEvent.click() from @vitest/browser/context
  • No shadow DOM string comparison available — assert on behavior (text content, attributes) instead
  • Empty describe() blocks error in Vitest ("No test found in suite")

Manual fixes needed beyond script

  • boolean-input — dynamic tag names require lit/static-html.js + manual fixture
  • file-dropzone — done() callback pattern → Promise-based
  • form-validation-message — nested describe inside it (Vitest forbids this)
  • avatar — shadowDom.toBe() → behavioral assertions
  • breadcrumb-item — fixture() calls missed by script
  • pagination — buttons re-render on click, need .querySelector('.active') + double updateComplete
  • Chai property chains (.to.false, .to.undefined) — silently pass but don't assert

Test plan

  • npx vitest run — 809 tests pass, 3 skipped, 80 files
  • CI passes (SonarCloud clean)
  • CI passes with all 3 browsers

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

2 similar comments
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard force-pushed the v2/spike/65042-vitest branch from ace783a to dcd57f7 Compare February 26, 2026 11:22
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard force-pushed the v2/spike/65042-vitest branch from dcd57f7 to e93a2cb Compare February 26, 2026 11:37
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

1 similar comment
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard force-pushed the v2/spike/65042-vitest branch from f4f335e to 9377f8b Compare February 26, 2026 15:20
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

5 similar comments
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

iOvergaard and others added 9 commits February 26, 2026 17:03
@iOvergaard @claude
test: spike Vitest browser mode as WTR replacement (AB#65042)
2f49afb 
Replace Web Test Runner with Vitest 4 browser mode + vitest-browser-lit
for the button and input component test suites (81 tests).

- Add Vitest config with Playwright chromium browser provider
- Convert button.test.ts and input.test.ts to vitest-browser-lit
- Add axe-core custom matcher (vitest-axe incompatible with browser mode)
- Add src/internal/test/a11y.ts helper for axe runs
- Remove WTR config and all @open-wc/testing dependencies
- Update CI workflow for Vitest
- Exclude vitest.setup.ts and vite.config.ts from build tsconfig
- Restrict test include to converted files only (spike scope)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
test: migrate all 80 component tests to Vitest 4 browser mode (AB#65042)
c1a82d2 
Full migration of test suite from @open-wc/testing + WTR to
vitest-browser-lit + Vitest 4. All 809 tests pass (3 skipped),
total runtime ~8.5s (down from ~45s with WTR).

Key changes:
- fixture() -> render().container.querySelector()
- Chai assertions -> Vitest expect()
- @open-wc/testing imports -> vitest-browser-lit
- UUITestMouse uses userEvent.click() from @vitest/browser/context
- done() callbacks converted to async/await with Promises
- Conversion script at scripts/convert-tests.mjs for reference

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
chore: remove one-time test conversion script
59251ed 
The convert-tests.mjs script was used during the WTR → Vitest
migration and is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
chore: revert unrelated USAGE-WITHOUT-BUNDLER.md change
5f63ea8 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
test: run all 3 browsers (Chromium, Firefox, WebKit) on CI
14508f5 
Matches the previous WTR behavior: Chromium only locally,
all 3 browsers on CI via process.env.CI check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
fix(test): round scroll values for Firefox subpixel precision
8d71159 
Firefox returns subpixel scrollTop/scrollLeft values (e.g. 42.316
instead of 42). Use Math.round() for cross-browser compatibility.

Also fix Assertion type augmentation generic for toHaveNoViolations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
fix(test): use vitest/browser instead of deprecated @vitest/browser/c…
31705dc 
…ontext

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
chore(test): suppress Lit dev mode stderr noise in test output
6d84502 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
test: convert version.test.ts to Vitest and silence UUI version warnings
1e3ea22 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard iOvergaard force-pushed the v2/spike/65042-vitest branch from 19b274e to 1e3ea22 Compare February 26, 2026 16:04
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 26, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard marked this pull request as ready for review February 27, 2026 07:48
Copilot AI review requested due to automatic review settings February 27, 2026 07:48
@iOvergaard iOvergaard added the internal Updates to dependencies, rewrite of files, cleanup, chores label Feb 27, 2026
@iOvergaard iOvergaard added the dependencies Pull requests that update a dependency file label Feb 27, 2026
Copilot AI reviewed Feb 27, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the test infrastructure from Web Test Runner (WTR) with @open-wc/testing to Vitest 4 browser mode. The migration converts all 80 test files (809 tests) to use vitest-browser-lit for rendering and Vitest's Jest-compatible assertions. Performance is improved by approximately 2x (warm runs: ~10s vs ~20s). The migration includes a custom axe-core accessibility matcher, replaces the oneEvent helper, and updates CI to run tests across Chromium, Firefox, and WebKit browsers.

Changes:

  • Replaces WTR + Mocha + Chai testing stack with Vitest 4 browser mode
  • Migrates all test files to use render() from vitest-browser-lit and Vitest assertions
  • Implements custom toHaveNoViolations() matcher for axe-core accessibility testing

Reviewed changes

Copilot reviewed 90 out of 92 changed files in this pull request and generated no comments.

Show a summary per file
File Description
web-test-runner.config.js Removed WTR configuration file
vitest.setup.ts New custom matcher for accessibility testing
vite.config.ts Added Vitest configuration with browser mode
tsconfig.json Updated types to include Vitest globals and browser
package.json Updated dependencies and test scripts
.github/workflows/tests.yml Updated CI to install Playwright browsers
src/internal/test/index.ts Simplified mouse helper using userEvent
src/internal/test/a11y.ts New axe-core wrapper for accessibility testing
src/internal/version.test.ts Sample migration showing before/beforeAll changes
src/components/*/**.test.ts Migrated all 80 component test files

iOvergaard and others added 2 commits February 27, 2026 08:54
@iOvergaard @claude
refactor(test): extract shared oneEvent helper to internal/test
154dc8a 
Removes 32 duplicate oneEvent() definitions from individual test
files and exports it from src/internal/test/index.ts instead.

Also narrows lint-staged tsc-files glob to *.element.ts to avoid
type-checking test files in pre-commit (they lack the vitest
setup context that provides custom matchers).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@iOvergaard @claude
chore: remove failure screenshots and add __screenshots__ to gitignore
ef9379e 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard enabled auto-merge (squash) February 27, 2026 12:07
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Feb 27, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 27, 2026

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

@iOvergaard iOvergaard merged commit a0d683a into main Feb 27, 2026
10 of 11 checks passed
@iOvergaard iOvergaard deleted the v2/spike/65042-vitest branch February 27, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@madsrasmussen madsrasmussen madsrasmussen approved these changes

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file internal Updates to dependencies, rewrite of files, cleanup, chores

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@iOvergaard @madsrasmussen